User interface for displaying internal state of autonomous driving system

ABSTRACT

A passenger in an automated vehicle may relinquish control of the vehicle to a control computer when the control computer has determined that it may maneuver the vehicle safely to a destination. The passenger may relinquish or regain control of the vehicle by applying different degrees of pressure, for example, on a steering wheel of the vehicle. The control computer may convey status information to a passenger in a variety of ways including by illuminating elements of the vehicle. The color and location of the illumination may indicate the status of the control computer, for example, whether the control computer has been armed, is ready to take control of the vehicle, or is currently controlling the vehicle.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 13/788,552, filed Mar. 7, 2013, which is a continuation of U.S.Pat. No. 8,433,470, which is a continuation of U.S. Pat. No. 8,352,110,which is a continuation of U.S. Pat. No. 8,260,482, which is acontinuation-in-part of U.S. Pat. No. 8,346,426, the entire disclosuresof which are hereby incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The invention relates generally to user interface applications forautonomous driving systems. More specifically, user interfaces fordisplaying the status of the autonomous driving system are provided.

2. Description of Related Art

Autonomous vehicles use various computing systems to transportpassengers from one location to another. Some autonomous vehicles mayrequire some initial input or continuous input from an operator, such asa pilot, driver, or passenger. Other systems, for example autopilotsystems, may be used only when the system has been engaged, thus theoperator may switch from a manual to an autonomous mode where thevehicle drives itself. These systems may be highly complicated andgenerally do not provide for a user friendly experience.

BRIEF SUMMARY OF THE INVENTION

A passenger in an automated vehicle may relinquish control of thevehicle to a control computer when the control computer has determinedthat it may maneuver the vehicle safely to a destination. The passengermay relinquish or regain control of the vehicle by applying differentdegrees of pressure, for example, on a steering wheel of the vehicle.The control computer may convey status information to a passenger in avariety of ways including by illuminating elements of the vehicle. Thecolor and location of the illumination may indicate the status of thecontrol computer, for example, whether the control computer has beenarmed, is ready to take control of the vehicle, or is currentlycontrolling the vehicle.

One aspect of the invention provides a vehicle including a plurality ofcontrol apparatuses including a braking apparatus, an accelerationapparatus, and a steering apparatus; a geographic position component fordetermining the current geographic location of the vehicle; a forceinput apparatus for identifying the approximate location and degree of aforce; memory for storing a detailed map information including roadway,traffic signal, and intersection information; and a processor. Theprocessor is operable to receive location information from thegeographic position component; receive input from the force inputapparatus; determine, from location information received from thegeographic position component and the stored map information, thecurrent geographic location of the vehicle; determine, based on thecurrent geographic location of the vehicle, whether the processor cancontrol the plurality of control apparatuses safely; and determine,based on input received from the force input apparatus, whether theprocessor has permission to control the plurality of controlapparatuses.

In one example, the processor is operable to control the plurality ofcontrol apparatuses when the current geographic location of the vehiclecorresponds to a roadway of the detailed roadway map. In anotherexample, the vehicle also includes a light emitting apparatus and theprocessor is also operable to illuminate the light emitting apparatus ifthe processor has determined that the processor has permission tocontrol the control apparatuses. In another example the force inputapparatus is associated with a threshold value, and the processor isfurther operable to determine that the processor has permission if aforce input on the force input apparatus is less than the thresholdvalue. In another example, the processor is also operable to control thecontrol apparatuses if the processor has determined that the processorhas permission to control the control apparatuses. In one alternative,the vehicle also includes a light emitting apparatus and the processoris also programmed to illuminate the light emitting apparatus while theprocessor is controlling the control apparatuses. In anotheralternative, the processor is also configured to determine, based on thecurrent geographic location of the vehicle, whether the processor can nolonger control the control apparatuses safely; and discontinue thecontrolling of the control apparatuses if the processor can no longercontrol the control apparatuses safely. In this regard, the vehicle mayalso include a light emitting apparatus and the processor may also beoperable to illuminate the light emitting apparatus if the processordiscontinues the controlling of the control apparatuses because theprocessor can no longer control the control apparatuses safely. Inanother alternative, the force input apparatus is associated with athreshold value and the processor is further operable to discontinuecontrol of the control apparatuses if a force input on the force inputapparatus is greater than the threshold value. In another alternative,the vehicle also includes a light emitting apparatus and the processoris also operable to illuminate the light emitting apparatus in a firstcolor if the processor can control the plurality of control apparatusessafely and the processor is not controlling the control apparatuses;control the control apparatuses and illuminate the light emittingapparatus in a second color if the processor has determined that theprocessor has permission to control the control apparatuses; andilluminate the light emitting apparatus in a third color if theprocessor discontinues the controlling of the control apparatusesbecause the processor can no longer control the control apparatusessafely.

Another aspect of the invention provides a method for controlling aplurality of control apparatuses of a vehicle including a brakingapparatus, an acceleration apparatus, and a steering apparatus. Themethod includes receiving location information from a geographicposition component for determining a current geographic location of thevehicle; receiving input from a force input apparatus identifying anapproximate location and degree of a force; determining, from thelocation information received from the geographic position component anddetailed map information, the current geographic location of thevehicle, the detailed map information including roadway, traffic signaland intersection information; determining, based on the currentgeographic location of the vehicle, whether a processor is operable tocontrol the plurality of control apparatuses safely; determining, basedon input received from the force input apparatus, whether the processorhas permission to control the plurality of control apparatuses; andcontrolling the control apparatuses if the processor has permission tocontrol the control apparatuses.

In one example, the force input apparatus is associated with a forcethreshold value and determining that the processor has permission isbased on whether a force input on the force input apparatus is less thanthe threshold value. In another example, the method also includesilluminating a light emitting apparatus if the processor is not operableto control the control apparatuses safely. In another example, themethod also includes illuminating a light emitting apparatus while theprocessor is controlling the control apparatuses. In another example, ifthe processor is not operable to control the control apparatuses safely,then discontinuing the controlling of the control apparatuses by theprocessor.

A further aspect of the invention provides a computer including memoryfor storing map information and a processor. The processor is operableto receive location information from a geographic position componentwhich determines a current geographic location of the vehicle; receiveinput from a force input apparatus which identifies an approximatelocation and degree of a force; determine, from the location informationreceived from the geographic position component and the stored mapinformation, the current geographic location of the vehicle; determine,based on the current geographic location of the vehicle, whether theprocessor is operable to control a plurality of control apparatusessafely; and determine, based on input received from the force inputapparatus, whether the processor has permission to control the pluralityof control apparatuses.

In one example, the plurality of control apparatuses control anautomobile. In another example, the plurality of control apparatusescontrol a boat. In another example, the plurality of control apparatusescontrol an airplane. In another example, the processor is also operableto illuminate light emitting apparatus in a first color if the processoris operable to control the plurality of control apparatuses safely andthe processor is not currently controlling the control apparatuses;control the control apparatuses if the processor has determined that theprocessor has permission to control the control apparatuses andilluminate the light emitting apparatus in a second color; andilluminate the light emitting apparatus in a third color if theprocessor discontinues the controlling of the control apparatusesbecause the processor can no longer control the control apparatusessafely.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a functional diagram of a system in accordance with an aspectof the invention.

FIG. 2 is an exemplary design of the interior of an autonomous vehiclein accordance with an aspect of the invention.

FIG. 3 is a view of the interior of an exemplary vehicle in accordancewith an aspect of the invention.

FIG. 4 is a view of the interior of an exemplary vehicle in accordancewith an aspect of the invention.

FIG. 5 is a view of the interior of an exemplary vehicle in accordancewith an aspect of the invention.

FIG. 6 is a view of the interior of an exemplary vehicle in accordancewith an aspect of the invention.

FIG. 7 is a view of the interior of an exemplary vehicle in accordancewith an aspect of the invention.

FIG. 8 is a view of the interior of an exemplary vehicle in accordancewith an aspect of the invention.

FIG. 9 is a view of the interior of an exemplary vehicle in accordancewith an aspect of the invention.

FIG. 10 is a view of the interior of an exemplary vehicle in accordancewith an aspect of the invention.

FIG. 11 is a view of the interior of an exemplary vehicle in accordancewith an aspect of the invention.

FIG. 12 is a view of the interior of an exemplary vehicle in accordancewith an aspect of the invention.

FIGS. 13A-C illustrate various states of an autonomous vehicle inaccordance with aspects of the invention.

FIG. 14 is a view of the interior of an exemplary vehicle in accordancewith an aspect of the invention.

FIG. 15 is a view of the interior of an exemplary vehicle in accordancewith an aspect of the invention.

FIG. 16 is a view of the interior of an exemplary vehicle in accordancewith an aspect of the invention.

DETAILED DESCRIPTION

Aspects, features and advantages of the invention will be appreciatedwhen considered with reference to the following description of exemplaryembodiments and accompanying figures. The same reference numbers indifferent drawings may identify the same or similar elements.Furthermore, the following description is not limiting; the scope of theinvention is defined by the appended claims and equivalents.

As shown in FIG. 1, an autonomous driving system 100 in accordance withone aspect of the invention includes a vehicle 101 with variouscomponents. The vehicle may be any type of vehicle including, but notlimited to, cars, trucks, motorcycles, busses, boats, airplanes, trams,golf carts, trains, and trolleys. The vehicle may have one or morecomputers, such as computer 110 containing a processor 120, memory 130and other components typically present in general purpose computers.

The memory 130 stores information accessible by processor 120, includinginstructions 132 and data 134 that may be executed or otherwise used bythe processor 120. The memory 130 may be of any type capable of storinginformation accessible by the processor, including a computer-readablemedium, or other medium that stores data that may be read with the aidof an electronic device, such as a hard-drive, memory card, ROM, RAM,DVD or other optical disks, as well as other write-capable and read-onlymemories. Systems and methods may include different combinations of theforegoing, whereby different portions of the instructions and data arestored on different types of media.

The instructions 132 may be any set of instructions to be executeddirectly (such as machine code) or indirectly (such as scripts) by theprocessor. For example, the instructions may be stored as computer codeon the computer-readable medium. In that regard, the terms“instructions” and “programs” may be used interchangeably herein. Theinstructions may be stored in object code format for direct processingby the processor, or in any other computer language including scripts orcollections of independent source code modules that are interpreted ondemand or compiled in advance. Functions, methods and routines of theinstructions are explained in more detail below.

The data 134 may be retrieved, stored or modified by processor 120 inaccordance with the instructions 132. For instance, although the systemand method is not limited by any particular data structure, the data maybe stored in computer registers, in a relational database as a tablehaving a plurality of different fields and records, XML documents orflat files. The data may also be formatted in any computer-readableformat. By further way of example only, image data may be stored asbitmaps comprised of grids of pixels that are stored in accordance withformats that are compressed or uncompressed, lossless (e.g., BMP) orlossy (e.g., JPEG), and bitmap or vector-based (e.g., SVG), as well ascomputer instructions for drawing graphics. The data may comprise anyinformation sufficient to identify the relevant information, such asnumbers, descriptive text, proprietary codes, references to data storedin other areas of the same memory or different memories (including othernetwork locations) or information that is used by a function tocalculate the relevant data.

The processor 120 may be any conventional processor, such as processorsfrom Intel Corporation or Advanced Micro Devices. Alternatively, theprocessor may be a dedicated device such as an ASIC. Although FIG. 1functionally illustrates the processor, memory, and other elements ofcomputer 110 as being within the same block, it will be understood bythose of ordinary skill in the art that the processor and memory mayactually comprise multiple processors and memories that may or may notbe stored within the same physical housing. For example, memory may be ahard drive or other storage media located in a housing different fromthat of computer 110. Accordingly, references to a processor or computerwill be understood to include references to a collection of processorsor computers or memories that may or may not operate in parallel.

Computer 110 may all of the components normally used in connection witha computer such as a central processing unit (CPU), memory (e.g., RAMand internal hard drives) storing data 134 and instructions such as aweb browser, an electronic display 142 (e.g., a monitor having a screen,a small LCD touch-screen or any other electrical device that is operableto display information), and user input (e.g., a mouse, keyboard,touch-screen and/or microphone).

Computer 110 may also include a geographic position component 144 todetermine the geographic location of the device. For example, computer110 may include a GPS receiver to determine the device's latitude,longitude and/or altitude position. Other location systems such aslaser-based localization systems, inertial-aided GPS, or camera-basedlocalization may also be used.

Computer 110 may also include other features, such as an accelerometer,gyroscope or other acceleration device 146 to determine the direction inwhich the device is oriented. By way of example only, the accelerationdevice may determine its pitch, yaw or roll (or changes thereto)relative to the direction of gravity or a plane perpendicular thereto.In that regard, it will be understood that a computer's provision oflocation and orientation data as set forth herein may be providedautomatically to the user, other computers, or both.

Computer 110 may also include an object detection component 148 todetect and identify the location and movement (e.g. relative speed) ofobjects such as other vehicles, obstacles in the roadway, trafficsignals, signs, etc. The detection system may include lasers, sonar,radar, cameras or any other such detection methods. For example, theobject detector may include an imaging device to identify the state of aparticular traffic signal as yellow or another color. In use, computer110 may use this information to instruct the braking system of thevehicle to apply the brakes.

Data 134 may include various types of information used by computer 110.Detailed map information 136 may include maps identifying lane lines,intersections, speed limits, traffic signals, buildings, signs, or othersuch information. For example, as will be described in more detailbelow, computer 110 may access detailed map information 136 in order todetermine whether computer 110 is in a location from which it maycompetently control aspects particular aspects of vehicle 101 such asdirection of travel, speed, acceleration, etc.

Computer 110 may also control status indicators 138, in order to conveythe status of computer 110 to a passenger of vehicle 101. For example,computer 110 may use visual or audible cues to indicate whether computer110 has been armed and is in control of the various systems of vehicle101, whether there are any errors, whether computer 110 has beendisarmed, etc. The various examples below describe visual cues whichinclude either text on an electronic display, illuminated portions ofvehicle 101, or both, although it will be understood that various othermethods of indications may also be used.

Computer 110 may include, or be capable of receiving information from,one or more touch sensitive input apparatuses 140. For example, computer110 may receive input from a user input apparatus and use thisinformation to determine whether a passenger is contacting, such as byholding or bumping, a particular portion of vehicle 110. The touchsensitive input apparatuses may be any touch sensitive input devicecapable of identifying a force, for example a force resistance tape maybe calibrated to accept or identify a threshold pressure input (such as10 grams of pressure) or a range of pressures (such as 5-20 grams ofpressure).

In one example, computer 110 may be an autonomous driving computingsystem capable of communicating with a vehicle's internal computer suchas computer 160. Computer 160 may be configured similarly to computer110, for example, including a processor 170, memory 172, instructions174, and data 176. Computer 110 may send and receive information fromthe various systems of vehicle 101, for example the braking 180,acceleration 182, signaling 184, and navigation 186 systems in order tocontrol the movement, speed, etc. of vehicle 101. In addition, whenengaged, computer 110 may control some or all of these functions ofvehicle 101 and thus be fully or merely partially autonomous. It will beunderstood that although various systems and computers 110 and 160 areshown within vehicle 101, these elements may be external to vehicle 101or physically separated by large distances.

FIG. 2 depicts an exemplary design of the interior of an autonomousvehicle. The autonomous vehicle may include all of the features of anon-autonomous vehicle, for example: a steering apparatus, such assteering wheel 210; a navigation display apparatus, such as navigationdisplay 215; and a gear selector apparatus, such as gear shifter 220.

Vehicle 101 may include one or more user input devices for inputtinginformation into the autonomous driving computer 110. For example, auser, such as passenger 290, may input a destination, (e.g. 123 OakStreet), into the navigation system using touch screen 217 or inputs219. In another example, a user may input a destination by identifyingthe destination audibly (e.g. by stating “De young museum” as in theexample of FIGS. 2 and 3).

Vehicle 101 may display information to a passenger in a number of ways.For example, vehicle 101 may be equipped with a display 225 fordisplaying information relating to computer 110. Vehicle 101 may alsoinclude a status indicating apparatus, such as status bar 230, toindicate the current status of vehicle 101. In the example of FIG. 2,status bar 230 displays “D” and “2 mph” indicating that the vehicle ispresently in drive mode and is moving at 2 miles per hour. As will bedescribed in more detail below various other audible and visualindicators may also be employed.

The navigation system may generate a route between the present locationof the vehicle and the destination. As shown in FIG. 3, afteridentifying the destination, navigation display 215 may indicate that aroute has been identified. Once the user has selected the identifiedroute, the navigation system may begin the route guidance, for as shownin FIG. 4.

When the passenger is ready to relinquish some level of control to theautonomous driving computing system or control computer, the user mayarm the control computer. For example, the passenger may press a buttonto arm computer 110. In another example, shown in FIG. 4, the passengermay arm computer 110 by manipulating a lever such as gear shifter 220into a particular position.

In response to the engagement action, various elements of vehicle 101may indicate that the control computer is now armed. For example, asshown in FIG. 5, display 225 may light up or display text indicatingthat computer 110 has been armed. Computer 110 may also provide audioindicating the same (e.g. “autonomous driving mode is engaged” or“cruise mode is engaged”). After arming computer 110, passenger 290 mayreturn his or her hands to steering wheel as shown in FIG. 6.

Once the control computer has been engaged, it must determine whether itmay competently control aspects of the vehicle. For example, computer110 may not be able to control aspects of vehicle 101 safely if vehicle101 is at a particular location which computer 110 is unable to identifybased on the geographic location of vehicle 101, if vehicle 101 is at aparticular location which is not sufficiently defined or described bydetailed map 136, or if computer 110 detects a large number of obstaclesin the vicinity of vehicle 101. As shown in FIG. 7, vehicle 101 islocated in a parking lot which, for example, computer 110 is unable tonavigate. Thus, computer 110 may display text information indicating thesame on status bar 230 (e.g. “Not available”).

Once the vehicle begins to move to new a location, the control computermay re-evaluate whether it may competently control aspects of thevehicle and display this information accordingly. For example, as shownby indicator bar 230 of FIG. 8, vehicle 101 has moved onto a roadway,computer 110 has initially determined it is unable to competentlycontrol aspects of vehicle 101. As shown in FIG. 9, once computer 110has determined that it is able to competently control aspects of vehicle101, indicator bar 230 may indicate that computer 110 is in the “ready”state or is able to take control of aspects of vehicle 101.

The control computer may use additional visual indicators to display thestatus of the control computer to a passenger. For example, computer 110may illuminate surfaces of vehicle 101 in a particular color (or variousshades of the color) to indicate the status of computer 110. Wherecomputer 110 is ready to take control of various aspects of vehicle 101,steering wheel portions 950 and 955 as well as indicator bar 230 mayilluminate gradually in a particular color, a first color. Theillumination may be immediately intense or alternatively graduallyincrease in intensity.

Once control computer is able to control the vehicle competently, thepassenger may relinquish control. For example, passenger 290 may releasehis or her hands from steering wheel 210. Various touch sensitive inputapparatuses 140 which may be disposed, for example, around or withinsteering wheel 210 may be used to identify the amount of pressure on thesteering wheel. If there is no more input or the input is below aparticular pressure threshold, computer 110 may determine that passenger290 has released steering wheel 210. Based on the information from thetouch sensitive input apparatuses, computer 110 may begin to controlvarious aspects of vehicle 101 as shown in FIG. 10.

As the control computer's changes, the control computer may again usevarious visual and audio indicators to convey this information to apassenger. For example, computer 110 may cause indicator bar 230,steering wheel portions 950 and 955, portions of display 215, and/oreven portions of steering wheel 210 to illuminate in a second color,different from the first color. Computer 110 may also inform thepassenger of the change using audio cues.

Once the indicators are illuminated such that they indicate that controlcomputer is in control of aspects of the vehicle, for example thosefunctions required for safely driving the vehicle between locations, theuser may now enjoy other pursuits. For example, as shown in FIG. 11,passenger 290 is able to use a cellular phone without fear that thedistraction will take away from the safety of the trip. The indicators,such as indicator box 230 and portions of steering wheel 210, allow thepassenger to feel safe in the knowledge that computer 110 is in controlof the vehicle.

If the passenger identifies an emergency situation, the passenger maytake control of the vehicle immediately. For example, passenger 290 maysee an obstacle which computer 110 has not identified, such as abicyclist or road construction. Without first disarming computer 110,passenger 290 may grip the steering wheel to return computer 110 to“ready mode” as shown in FIG. 9. The impact of passenger 290's hand orhands on steering wheel 210 may be received by the various touchsensitive input apparatuses 140 of steering wheel 210. Computer 110 mayreceive this information, determine that the passenger would like totake control, and return to ready mode. This allows the user to feelconfident that he or she may take control of vehicle 101instantaneously.

In order to prevent returning the control computer to ready mode inevery case of contact, the touch sensitive input apparatuses may becalibrated to prevent accidental transferring control from the controlcomputer to the passenger. As shown in FIG. 12, passenger 290 mayaccidentally bump steering wheel 210 and place computer 110 into readymode. However, it could be dangerous to allow computer 110 to relinquishcontrol to the passenger, for example if the passenger is actuallyasleep or not paying attention. Thus, the touch sensitive inputs mayonly accept certain types of input. For example, the contact may berequired to be in a particular location along steering wheel 210, forexample, the top or bottoms surface of steering wheel 210 or both. Inanother example, the contact may be required to be over a particulartime or pressure threshold as described above. In a further example, theuser may be required to contact the steering wheel with two hands ratherthan one. Alternatively, the touch sensitive inputs may always send theinputted information to computer 110 which may determine whether therequired contact or contacts have been met.

Control computer may use pressure sensing devices at other locations ofthe vehicle in order to determine whether the passenger is ready or ableto relinquish or regain control. For example, touch sensitive inputapparatuses 140 may also (or alternatively) be located on the brakeand/or acceleration pedals. If computer 110 has been armed and thepassenger applies some threshold pressure to the pedals, computer 110may enter ready mode. Thus, if computer 110 is controlling vehicle 101and passenger 290 applies the brakes, computer 110 will return to readymode. Or if the passenger has released the steering wheel, but continuesto apply pressure to the pedals, computer 110 may continue in readymode, but request that the passenger grip the steering wheel.

In the event of an emergency situation identified by the controlcomputer, it may immediately convey this information to the passenger ina variety of ways. For example, if computer 110 is no longer able tocontrol vehicle 101 safely, the illuminated indicators described abovemay change to a third color, to inform the user of the immediate need totake control of the steering wheel. Computer 110 may also send audioalerts to passenger 290 stating that there is an emergency and thepassenger is required to take control of the steering wheel.

Because of the changes in the status, for example between FIGS. 9 and10, it would be beneficial to illuminate these elements in differentcolors to indicate the differing statuses of the control computer. Thus,the change in the illumination may clearly convey the change in thestatus of computer 110. As shown in FIG. 13, computer 110 may usevarious color cues to identify the status of vehicle 101.

FIG. 13A is a summary exemplary colors and status indicators wherecomputer 110 has been armed. If the passenger is gripping the steeringwheel and applying, or not applying, a force on the brake oracceleration pedal, computer 110 may be in ready mode and theillumination may be blue in color as shown in blocks 1360 and 1362. Ifthe passenger releases the grip on the steering wheel, but continues toapply pressure on one of the pedals, computer 110 may continue in readymode and the illumination may be blue in color as shown in block 1364.Computer 110 may also display and/or sound text requesting that thepassenger grip the wheel as described above. Where the passenger hasreleased the steering wheel, is not applying a force on a pedal, andcomputer 110 has actually taken control, the illumination may be greenin color as shown in block 1366.

FIG. 13B is a summary exemplary colors and status indicators wherecomputer 110 has been armed, but computer 110 is unable to take control(or continue controlling) vehicle 101. If the passenger is gripping thesteering wheel and applying, or not applying, a force on the brake oracceleration pedal, computer 110 may display text indicating that it is“not available” and no illumination may be used as shown in blocks 1370and 1372. If the passenger releases the grip on the steering wheel, butcontinues to apply pressure on one of the pedals, computer 110 maydisplay text indicating that it is “not available” and no illuminationmay be used as shown in block 1374. Computer 110 may also display and/orsound text requesting that the passenger grip the wheel as describedabove. If the passenger has released the steering wheel, is not applyinga force on a pedal and computer 110 was not previously in control,computer 110 may again display and/or sound text requesting that thepassenger grip the wheel as described above as shown in block 1376. Ifthe passenger has released the steering wheel, is not applying a forceon a pedal and computer 110 was previously in control, the illuminationmay be red, indicating the emergency situation as shown in block 1377.Again computer 110 may display and/or sound text requesting that thepassenger take control of the vehicle.

FIG. 13C indicates exemplary colors and status indicators where computer110 has not been armed. If computer 110 has not been armed, computer 110need not determine whether there is any pressure or contact on thesteering wheel or pedals. Thus, no illumination is required in any ofblocks 1380, 1382, 1384, or 1386 as the status of computer 110 is always“off.”

It will be understood that the particular combinations of color andelements selected for illumination are merely exemplary and there is nolimit to the number of combinations which may be utilized.

The passenger may also disarm the control computer while the vehicle ismoving. For example, as shown in FIG. 14, when passenger 290 is ready todisarm computer 110, the passenger may place his or her hands onsteering wheel 210 and gear shifter 220. Moving the gear shifter 220into another mode as shown in FIG. 15, such as drive, may disarmcomputer 110. Thus, as shown in FIGS. 15 and 16, status bar 230 may nowdisplay “D” indicating that computer 110 is no longer armed and vehicle101 is in drive. In addition, the illumination of indicator bar 230,steering wheel portions 950, 955, portions of display 215, and portionsof steering wheel 210 may fade out, for example, to black, furtherindicating that computer 110 is no longer armed.

The control computer may be configured to take control of variousaspects of the vehicle to various degrees. For example, computer 110 maytake complete control of the systems of vehicle 101 and control allaspects necessary to do so, such as braking, signaling, and accelerationsystems. In another example, computer 110 may only control the positionof vehicle 101, such as by maintaining vehicle 101's position within alane of traffic.

The control computer may also to determine whether the driver is able tocontrol the vehicle and, if not, take emergency control. For example,computer 110 may be used as a safety mechanism to protect passenger 290.In addition to receiving inputs from the touch sensitive inputs,Computer 110 may use cameras to scan the passenger's eyes and/or otherfeatures. If computer 110 determines that the eyes are closed or thebody is slumped over (e.g. the passenger is sleeping or under theinfluence of drugs or alcohol), computer 110 may take emergency controlof vehicle 101 and use audible alerts to wake or otherwise get theattention of the driver.

As these and other variations and combinations of the features discussedabove can be utilized without departing from the invention as defined bythe claims, the foregoing description of exemplary embodiments should betaken by way of illustration rather than by way of limitation of theinvention as defined by the claims. It will also be understood that theprovision of examples of the invention (as well as clauses phrased as“such as,” “e.g.”, “including” and the like) should not be interpretedas limiting the invention to the specific examples; rather, the examplesare intended to illustrate only some of many possible aspects.

The invention claimed is:
 1. A method comprising: receiving, by one ormore processors, input at a user input apparatus associated with asteering wheel of a vehicle indicating that a user is ready torelinquish control of the vehicle to a control computer; in response tothe input, illuminating, by the one or more processors, a portion of thevehicle in a first color to indicate that the control computer isengaged and ready to control the vehicle in an autonomous driving mode;and when the control computer begins to control the vehicle in theautonomous driving mode, illuminating, by the one or more processors,the portion of the vehicle in a second color, different from the firstcolor, to indicate that the control computer is controlling the vehiclein the autonomous driving mode.
 2. The method of claim 1, furthercomprising: receiving second input at a second user input deviceassociated with a brake pedal of the vehicle; and wherein the portion ofthe vehicle is illuminated in the first color when the input and thesecond input are received in conjunction with one another.
 3. The methodof claim 1, further comprising: receiving second input at a second userinput device associated with a brake pedal of the vehicle prior toreceiving the input; and in response to the second input, illuminating,by the one or more processors, the portion of the vehicle in the firstcolor; in response to the second input, providing a notificationrequesting a user to complete the input; wherein the input is receivedafter the notification is provided.
 4. The method of claim 3, whereinthe notification includes an audible message requesting the user to gripthe steering wheel.
 5. The method of claim 3, wherein the notificationincludes displaying a text message requesting the user to grip thesteering wheel.
 6. The method of claim 1, wherein the portion of thevehicle is associated with a steering wheel of the vehicle.
 7. Themethod of claim 1, further comprising: in response to the input,determining whether the control computer is able to control the vehiclesafely; when the control computer is not able to control the vehiclesafely, not illuminating the portion of the vehicle in the first colorto indicate that the control computer is engaged and ready to controlthe vehicle in an autonomous driving mode.
 8. The method of claim 1,further comprising: determining whether the control computer is ready tocontrol the vehicle in the autonomous driving mode, wherein the portionof the vehicle is illuminated in the first color only when the controlcomputer is ready to control the vehicle in an autonomous driving mode.9. A system comprising one or more processors configured to: receiveinput at a user input apparatus associated with a steering wheel of avehicle indicating that a user is ready to relinquish control of thevehicle to a control computer; in response to the input, illuminate aportion of the vehicle in a first color to indicate that the controlcomputer is engaged and ready to control the vehicle in an autonomousdriving mode; and when the control computer begins to control thevehicle in the autonomous driving mode, illuminate the portion of thevehicle in a second color, different from the first color, to indicatethat the control computer is controlling the vehicle in the autonomousdriving mode.
 10. The system of claim 9, wherein the one or moreprocessors are further configured to: receive second input at a seconduser input device associated with a brake pedal of the vehicle; andwherein the portion of the vehicle is illuminated in the first colorwhen the input and the second input are received in conjunction with oneanother.
 11. The system of claim 9, wherein the one or more processorsare further configured to: receive second input at a second user inputdevice associated with a brake pedal of the vehicle prior to receivingthe input; and in response to the second input, illuminate, by the oneor more processors, the portion of the vehicle in the first color; inresponse to the second input, provide a notification requesting a userto complete the input; wherein the input is received after thenotification is provided.
 12. The system of claim 9, wherein thenotification includes an audible message requesting the user to grip thesteering wheel.
 13. The system of claim 9, wherein the notificationincludes displaying a text message requesting the user to grip thesteering wheel.
 14. The system of claim 9, wherein the portion of thevehicle is associated with a steering wheel of the vehicle.
 15. Thesystem of claim 9, wherein the one or more processors are furtherconfigured to: in response to the input, determine whether the controlcomputer is able to control the vehicle safely; when the controlcomputer is not able to control the vehicle safely, not illuminating theportion of the vehicle in the first color to indicate that the controlcomputer is engaged and ready to control the vehicle in an autonomousdriving mode.
 16. The system of claim 9, wherein the one or moreprocessors are further configured to: determine whether the controlcomputer is ready to control the vehicle in the autonomous driving mode,wherein the portion of the vehicle is illuminated in the first coloronly when the control computer is ready to control the vehicle in anautonomous driving mode.
 17. The system of claim 1, further comprisingthe vehicle.
 18. A non-transitory computer-readable storage medium onwhich computer readable instructions of a program are stored, theinstructions, when executed by one or more processors, cause the one ormore processors to perform a method, the method comprising: receivinginput at a user input apparatus associated with a steering wheel of avehicle indicating that a user is ready to relinquish control of thevehicle to a control computer; in response to the input, illuminating aportion of the vehicle in a first color to indicate that the controlcomputer is engaged and ready to control the vehicle in an autonomousdriving mode; and when the control computer begins to control thevehicle in the autonomous driving mode, illuminating the portion of thevehicle in a second color, different from the first color, to indicatethat the control computer is controlling the vehicle in the autonomousdriving mode.
 19. The medium of claim 18, wherein the method furthercomprises: receiving second input at a second user input deviceassociated with a brake pedal of the vehicle; and wherein the portion ofthe vehicle is illuminated in the first color when the input and thesecond input are received in conjunction with one another.
 20. Themedium of claim 19, wherein the method further comprises: receivingsecond input at a second user input device associated with a brake pedalof the vehicle prior to receiving the input; and in response to thesecond input, illuminating, by the one or more processors, the portionof the vehicle in the first color; in response to the second input,providing a notification requesting a user to complete the input;wherein the input is received after the notification is provided.